DOCUMENT RESUME 



ED 464 403 



EA 031 639 



AUTHOR 

TITLE 



PUB DATE 
NOTE 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



Boden, Andrea; Archwamety, Teara; McFarland, Max 
Programmed Instruction in Secondary Education: A 
Meta-Analysis of the Impact of Class Size on Its 
Effectiveness . 

2000-03-00 

24p . ; Paper presented at the Annual Meeting of the National 
Association of School Psychologists (New Orleans, LA, March 
28 -April 1, 2000) . 

Reports - Research (143) -- Speeches/Meeting Papers (150) 

MF01/PC01 Plus Postage. 

Academic Achievement; Class Size; Comparative Analysis; 
Conventional Instruction; Instructional Effectiveness; *Meta 
Analysis; *Programmed Instruction; Secondary Education 



ABSTRACT 



This review used meta-analytic techniques to integrate 
findings from 30 independent studies that compared programmed instruction to 
conventional methods of instruction at the secondary level. The meta-analysis 
demonstrated that programmed instruction resulted in higher achievement when 
compared to conventional methods of instruction (average ES=.40). No 
significant correlation was found between class size and effect size (r=.097, 
p>= . 05 ) . The most important aspect of this meta-analysis is that with this 
virtually zero correlation, it indicates that programmed instruction 
maintains its effectiveness over conventional methods of instruction 
regardless of class size. An appendix contains a data-coding form. (Contains 
42 references, 1 table, and 5 figures.) (Author /RT) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



Programmed Instruction 1 



ro 

O 

^r 

MD 



Q 



ffl 



Programmed Instruction in Secondary Education: A Meta-Analysis 



of the Impact of Class Size on Its Effectiveness 



Andrea Boden 
Teara Archwamety 
Max McFarland 



University of Nebraska at Kearney 



A Paper Presented (Poster Session) at the 2000 Annual Meeting of 
National Association of School Psychologists (NASP) 

New Orleans, March 28-April 1, 2000 



BEST COPY AVAILABLE 



cr~ 

CO 

^9 



CP 0 

c ERLC 



ld 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION 
/ CENTER (ERIC) 

□rThis document has been reproduced as 
received from the person or organization 
originating it. 

□ Minor changes have been made to 
improve reproduction quality. 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 



P[ , I 3qq( 






• Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



2 



Programmed Instruction 2 

Abstract 

This review used meta-analytic techniques to integrate findings from 30 independent studies that 
compared programmed instruction to conventional methods of instruction at the secondary level. 
The meta-analysis demonstrated that programmed instruction resulted in higher achievement 
when compared to conventional methods of instruction (average ES=.40). No significant 
correlation was found between class size and effect size (r=-.097, p>.05). The most important 
aspect of this meta-analysis is that with this virtually zero correlation, it indicates that 
programmed instruction maintains its effectiveness over conventional methods of instruction 
regardless of class size. 
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Programmed Instruction in Secondary Education: A Meta-Analysis 
of the Impact of Class Size on Its Effectiveness 
In 1953 following an observation of a fourth grade mathematics class, B.F. Skinner built 
the first teaching machine, in an effort to address what he perceived as being the violation of two 
fundamental principles in the classroom (Smith & Woodward, 1996). These two violations 
included students failing to receive immediate feedback on their work (lack of reinforcement) 
and expectations of all students to work at the same pace regardless of ability and preparation. 
These machines made a significant impact on teaching and were implemented in many 
classroom settings in the late 1950’s and throughout the 1960’s. Programmed instruction prepared 
schools for teaching technologies that would follow. 

A large amount of research was conducted on programmed instruction by comparing 

examination scores of groups taught by programmed and conventional methods. These studies 

) 

were completed on groups of students in all levels of education, where a variety of research 
designs were used and a variety of results were obtained. Many variables were analyzed to 
determine the impact they had on the effectiveness of programmed instruction. Inconsistencies in 
the research findings have prevented any clear cut pictures to be drawn regarding programmed 
instruction, however some patterns seem to exist. Elaboration will follow. 

The research indicates that programmed instruction is more effective for social science 
classes than for mathematics or science. Zoll (1969) reported that only three out 
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of thirteen studies reviewed, found programmed instruction to be significantly superior to 
conventional methods of instruction for mathematics. An extensive review by Kulik, Schwalb, 
and Kulik (1982) summarized results from 47 comparisons of programmed and conventional 
instruction and found the average Effect Size in the social sciences (.57), in mathematics (-.01), 
and in science (.11). Briggs and Angell (1966) found favorable results in only two out of 14 

studies reviewed in the field of mathematics. 

Hartley (1978), and Kulik, Cohen, and Ebeling (1980) found that more recent studies of 
programmed instruction have shown stronger effects, as the correlation between the year and 
size of effect was .31. In a more recent review of research Kulik, Schwalb, and Kulik (1982) 
found the following Effect Sizes for the five year periods reviewed. -.01 for studies published in 
the years 1961-65; .14 for the 19 studies published in the years 1966-70; and .28 for the seven 
studies published in the years 1971-75. 

Student ability is another variable that has been shown to influence achievement from 
programmed instruction. General intelligence appears to be related more to the pace with which 
a student goes through a program than to achievement according to Glaser, Reynolds, and 
Fullick (1967). Cassel and Ullom (1962) found programmed instruction to result in a highly 
significant amount of learning in mathematics for a group of high ability high school students. A 
large portion of the research reviewed by Zoll (1969) found programmed instruction to be as 
effective as conventional methods of instruction for students of average to high ability, and 
found conventional methods to be superior to programmed instruction for low ability students. 
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Briggs and Angell (1966) concluded that the more closely programmed instruction resembles 
conventional methods of instruction the higher will be its correlation with intelligence. 

Reviews of research seem to indicate that programmed instruction is more effective in the 
upper grade levels. Hartley (1978) reviewed 40 research articles on programmed instruction at 
the elementary and secondary level and found that programmed instruction raised student 
achievement by .11 standard deviation units. Kulik, Cohen, and Ebeling (1980) integrated results 
from 57 studies at the college level to find that programmed instruction raised achievement by 
.25 standard deviation units. When comparing Effect Sizes within the secondary level Kulik, 
Schwalb, and Kulik (1982) found a range of .01 for 7th grade (five studies reviewed) and .33 for 
12th grade (3 studies reviewed). 

Besides the effect of programmed instruction on student achievement, student attitudes 
toward programmed instruction have been studied as well. Hartley (1974) in a review of 
literature found that 70-90% of the students involved in short-term experiences with 
programmed instruction responded favorably. Mellish and Bostow (1975) reported that students 
responded favorably and that programmed instruction seemed to improve the student’s attitude 
toward studying. Okunrotifa (1975) found that programmed instruction produced more favorable 
attitudes towards map reading in geography than conventional texts covering the same span of 
information as the programs. These results imply that teachers may select programmed 
instruction as a teaching method to promote favorable attitudes toward school curriculum. 

The time required to present material has been found to be shorter by programmed 
instruction than conventional instruction, (Hughes & McNamara, 1961). Fusfeld and 
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Jump (1966) reported that students who studied using programmed instruction spent much less 
time than students using the conventional method, yet there was no significant difference in their 
test scores. Other studies have also indicated reduced time for the learner (Jensen, 1949; Porter, 
1959). 

Extensive research has been completed, that examined the factor of class size and 
achievement, and this research (Glass & Smith, 1979) determined that achievement decreased as 
class size increased when conventional methods of instruction were used (see Figure 1). 
However, none of the above studies have investigated the class size factor and its effect on the 
Effect Size of Programmed Instruction. The present study attempts to remedy that situation. This 
article applies Glass’s meta-analytic methodology to evaluations of programmed instruction at 
the secondary level. It thus serves as a supplement to previous work with other factors that 
influence the effectiveness of programmed instruction. This article focuses on three research 
questions. First, what is the average Effect Size for programmed instruction vs. conventional 
methods of instruction at the secondary school level? Second, what is the average class size for 
studies reviewed? Third, what is the correlation between class size and Effect Size for 
Programmed Instruction? A positive correlation would indicate a situation such as shown in 
Figure 2a. A zero correlation would indicate a situation such as shown in Figure 2b. A negative 
correlation would indicate a situation such as shown in Figure 2c. 

Method 

This section describes the procedures that were used in locating studies, coding study 

r 

features, and quantifying outcomes of studies. 
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Locating Studies 

The first step of this meta-analysis was to collect a large number of studies for 
secondary students that compare the effectiveness of programmed instruction to conventional 
methods of instruction. The main sources for these studies were three library databases. These 
data bases included (a) Comprehensive Dissertation Abstracts; (b) ERIC, a data base on 
educational materials from the Educational Resources Information Center, consisting of two files 
Research in Education and Current Index to Journals in Education; (c) Psychological abstracts. 
The bibliographies in articles located through computer searches provided a second source of 
studies for meta-analysis. 

A set of guidelines was used when reducing a pool of approximately 200 titles to a final 
group of approximately 30. To be included in the final sample they needed to meet four criteria. 
First, the studies took place in actual secondary classrooms. Second, studies need to report on 
quantitatively measured outcomes in both programmed instruction and conventional classes. 
Studies without control groups were excluded. Third, studies have to be free from crippling 
methodological flaws. Studies were not included if treatment and control groups clearly differ in 
aptitude. Nor were they included where criterion tests are taught unfairly to one of the 

comparison groups. Fourth, studies have to report the number of students in each class being 

c 

compared. 

Guidelines were used to maximize independence among comparisons and prevent studies 
from being counted more than once. When several papers report the same comparison, the most 
complete report was used for analysis. 
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Instrument (Data Coding Form) 

The Data Coding Form (see Appendix) included (A) Reference, (B) Subject Matter, 

(C) Grade Level, (D) Length of Study, (E) Class Size, (F) Effect Size. The outcomes 
described in the studies are for student performance on final examinations and 
performance on retention examinations. Effect Size for each of the outcomes were 
calculated by using procedures described by McGaw and Glass (1980). The Effect Sizes 
were based on group averages for the two instructional outcomes. For the index of the 
size of difference between group averages, Glass’s Effect Size was used. It is defined as 
the difference between the treatment and control means, divided by the standard 
deviation of the control group (Glass, 1976). In all of the 30 studies, the ES was 
calculated directly from means and standard deviations provided in the reports. 

Results 

A total of 30 studies contained information regarding class size and achievement. 

Class Size 

As shown in Table 1 the average class size for the 30 studies used in this meta-analysis was 
33. The standard deviation for class size was 23. 

Achievement 

Table 1 shows that the average value of Effect Size in the 30 studies was .40. The standard 
deviation of Effect Size was .80. Therefore programmed instruction raised student achievement 
when compared to conventional methods of instruction at the secondary level. 
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Class Size and Effect Size 

« 

The Pearson Product Moment Correlation between Effect Size and class size found there to 
be no significant correlation. The correlation coefficient was -.097, g>.05. This indicates that 
effect size was not significantly affected by class size (see Figure 2 & 2b). 

Discussion 

Extensive prior research (Glass & Smith, 1979) found that there is a fixed line that represents 
a decrease in achievement as class size increases when conventional methods of instruction are 
used. This meta-analysis demonstrated two additional points. The first is that programmed 
instruction increased achievement among students when compared to conventional methods of 
instruction (ES=.40). Second the nonsignificant correlation (-.097, p>.05) between class size and 
effect size indicates that programmed instruction maintains its effectiveness over conventional 
methods of instruction regardless of class size (see Figure 2 & 2b). These findings would suggest 
that programmed instruction would be the instructional method of choice when teaching classes 
of various sizes at the secondary level. 

The effect size of .40 found in this meta-analysis is somewhat surprising, given the results of 
a previous meta-analysis (Kulik, Schwalb, & Kulik, 1982) regarding programmed instruction at 
the secondary level, which found an average effect size of .08 when 48 studies were reviewed. 
However, previous research (Kulik, Schwalb, & Kulik, 1982) did find that the more recent 
(1971-1975) use of programmed instruction was more effective than earlier (1961-1965) uses of 
programmed instruction, which could explain the difference in the average effect size of these 
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two meta-analysis as 19 of the 30 studies used in this meta-analysis involved studies completed 
more recently than 1965. Another factor that may have contributed to these findings is that 17 of 
the 30 studies used in this meta-analysis involved 11th and 12th grade students and the other 
meta-analysis (Kulik, Schwalb, & Kulik, 1982) found an average effect size of .35 with 11th and 
12th grade students when compared to an average effect size of .02 with 7th through 10th grade 
students. This meta-analysis established that programmed instruction was more effective than 
conventional methods of instruction and this may be attributed to many of these studies 
involving more recent research and students in the 1 1th and 12th grade. 

The second point demonstrated by this meta-analysis involved there being no significant 
correlation between class size and effect size. The fact that programmed instruction was able to 
maintain its effectiveness over conventional methods of instruction regardless of class size, may 
be attributed to the sequential, work at your own pace, and reinforcement aspects of programmed 
instruction. As these aspects of programmed instruction seem to require more independence on 
the part of the student and therefore are less dependent on the teacher which makes the teacher to 
student ratios less important. This may explain how programmed instruction is able to maintain 
its effectiveness over conventional methods of instruction across various class sizes. 

When interpreting the results of this meta-analysis two points should be kept in mind. First 
studies were located in the library by using the search term "programmed instruction." This is 
consistent with how previous researchers located studies, however, there may be a difference in 
the way programmed instruction is defined and utilized to present material to students. 
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Therefore programmed instruction was what teachers in previous studies said it was. Second it is 
important to keep in mind that a meta-analysis is a method of using previous studies to determine 
themes in past research, rather than making predictions for the future. 
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A) Reference 



B) Subject Matter 

C) Grade Level 

D) Length of Study 

E) Class Size 

F) Effect Size 
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Table 1 

Class Size and Effect Size for Secondary Students When Programmed Instruction Was 
Used. 
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Figure 1 . Consistent regression lines for the regression of achievement (expressed in percentile 
ranks) onto class size for elementary and secondary grades (Glass 8c Smith, 1979). 
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Class Size 1 

Figure 2 . Consistent regression lines for the regression of achievement onto class size for 
studies involving secondary students, where either Programmed Instruction or Conventional 
Instruction was used. 
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Figure 2a. Positive correlation between class size and Effect Size. 
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Figure 2b. Zero correlation between class size and Effect Size. 
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Figure 2c. Negative correlation between class size and Effect Size. 
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